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ABSTRACT 

Objective: To investigate the extent to which risk 
scales were used for the assessment of self-harm by 
emergency department clinicians and mental health 
staff, and to examine the association between the use 
of a risk scale and measures of service quality and 
repeat self-harm within 6 months. 
Design: Observational study. 
Setting: A stratified random sample of 32 hospitals in 
England. 

Participants: 6442 individuals presenting with self- 
harm to 32 hospital services during a 3-month period 
between 2010 and 2011. 

Outcomes: 21 -item measure of service quality, repeat 
self-harm within 6 months. 
Results: A variety of different risk assessment tools 
were in use. Unvalidated locally developed proformas 
were the most commonly used instruments (reported in 
n=22 (68.8%) mental health services). Risk assessment 
scales were used in one-third of services, with the SAD 
PERSONS being the single most commonly used scale. 
There were no differences in service quality score 
between hospitals which did and did not use scales as a 
component of risk assessment (median service quality 
score (IQR): 14.5 (12.8, 16.4) vs 14.5 (11.4, 16.0), 
U=1 21 .0, p=0.90), but hospitals which used scales had 
a lower median rate of repeat self-harm within 6 months 
(median repeat rate (IQR): 18.5% vs 22.7%, p=0.008, 
IRR (95% CI) 1.18 (1.00 to 1.37). When adjusted for 
differences in casemix, this association was attenuated 
(IRR=1.13,95%CI (0.98 to 1.3)). 
Conclusions: There is little consensus over the best 
instruments for risk assessment following self-harm. 
Further research to evaluate the impact of scales 
following an episode of self-harm is warranted using 
prospective designs. Until then, it is likely that the 
indiscriminant use of risk scales in clinical services will 
continue. 



Correspondence to 

L Quinlivan; 

leah.quinlivan@manchester. 
ac.uk 



BACKGROUND 

Hospital services play an important role in 
engaging people who self-harm and treat 
approximately 220 000 episodes of self-harm 
in England annually/ Self-harm is associated 
with a high risk of suicide and other adverse 



Strengths and limitations of this study 



This study involved a large national sample of 
patients and services and it is likely that our find- 
ings will be generalisable to the rest of England 
and applicable to managed healthcare settings in 
other countries. 

We gathered responses from staff closely 
involved in self-harm service provision, and 
obtained copies of the risk tools. 
The study was observational in design and we 
cannot therefore infer causation. 
We only considered repeat episodes of self-harm 
presenting to the study hospitals, rather than 
community episodes or episodes presenting to 
other centres. 



outcomes, and good quality care has the 
potential to contribute significantly to suicide 
prevention.^ Psychosocial assessment is a key 
component of management.^ Clinical guide- 
lines recommend a psychosocial assessment 
on presentation to the emergency depart- 
ment in order to gauge a person's mental 
state and willingness to remain for further 
assessment.^ 

Assessments by mental health staff typically 
take place at a later stage and involve a com- 
prehensive evaluation of an individual's per- 
sonal situation, history of self-harm, family 
history and mental state. ^ ^ The evaluation of 
risks (factors which may increase the likeli- 
hood of adverse outcomes) and patient 
needs (resulting, eg, from mental or physical 
illness or difficult social circumstances) are 
particularly important.^ 

Clinical assessments are social interactions 
between clinical staff and patients. Some 
large quantitative studies have suggested 
that the process may itself have a protective 
effect against future suicidal behaviour^ 
Assessments may be most helpful when used as 
an opportunity for clinicians to engage with 
patients and when patients have their need for 
help legitimised by the process. Tailored 
management plans can be developed, laying 
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the foundation for longer term care 7 ^ In contrast, other 
studies found that clinical assessments may sometimes 
encourage clinicians to disengage with patients and there- 
fore negatively impact the therapeutic relationship/^ 

Risk assessment tools consisting of brief checklists of 
key risk factors, symptoms or antecedents are often 
regarded as core components of psychosocial assess- 
ments, but evidence for their effectiveness is 
limited.^ Some guidelines advise against the use of 
these tools to determine clinical management.^ Others 
recommend that clinicians should only use scales that 
have undergone psychometric testing^^ — for example, 
tested for their reliability, predictive ability and diagnos- 
tic accuracy, as well as construct, internal and external 
validity/^ 

A small number of descriptive studies have investigated 
the use of risk assessment tools in UK mental health ser- 
vices. Risk assessment tools vary widely in their 
type, structure and content,^^ with clinicians some- 
times using locally developed and untested tools. 
The use of risk tools may also differ by care setting. For 
example, emergency department service pressures often 
necessitate rapid forms of assessment for physical risk, 
and scales are commonly used.^^ 

There have been few studies on the use of risk scales 
across self-harm services. This is perhaps surprising 
given the high risk of suicide and other adverse out- 
comes in patients who self-harm. To our knowledge, no 
studies until now have investigated the association 
between the use of risk scales and service-level outcomes 
such as the quality of care or the rate of repetition of 
self-harm. 

Using data from a study of 32 hospitals in England,^^ 
we aimed to investigate the use of risk scales following 
self-harm in National Health Service (NHS) services. We 
included emergency departments and specialist mental 
health treatment settings in order to capture initial and 
subsequent risk assessments. 

Specifically, the objectives of this study were: 

1. To investigate the extent to which risk scales were 
used for the assessment of self-harm by emergency 
department clinicians and mental health staff. 

2. To examine the association between use of a risk 
scale and service-level outcomes, such as service 
quality and repeat self-harm within 6 months. 



METHOD 

Recruitment and study procedure 

For this observational study, we used data from a nationally 
representative stratified random sample of 32 hospitals in 
England.^^ The methodological details are described else- 
where,^^ but in brief, the sample was stratified to include 
small and large hospitals, and hospitals with low self-harm 
and high self-harm admission rates. The original sample 
was also stratified to ensure that four hospitals were 
selected within each of the eight former health regions. 



Only hospitals with an emergency department were 
included in the sample. 

Half of the hospitals were teaching hospitals (16/32). 
Fifteen of the hospitals served urban areas, 7 rural and 
10 urban and rural equally. The number of beds ranged 
from 300 to 1865. The hospital catchment population 
(based on hospital self-report) ranged from 120 000 to 
750 000. Specialist self-harm teams (ie, any liaison psy- 
chiatric service with at least one member of staff located 
within the emergency department) were in place in the 
majority of hospitals (29/32). Most of the assessments 
were carried out by a mental health nurse, including 
mental health liaison nurses and specialist self-harm 
teams (3109/4075), followed by a psychiatrist (799/ 
4075) and 167 by another mental health professional 
(eg, social worker). The proportion of self-harm epi- 
sodes given a psychosocial assessment by mental health 
staff varied from 24% to 88%, with a median rate of 
58%. 

Key emergency department and psychiatric staff (eg, 
consultants in emergency medicine and liaison psych- 
iatry, mental health managers and mental health 
nurses) were interviewed at each site about self-harm 
service provision. As part of the interview, we asked 
about the methods used for assessing risk and for copies 
of the risk measures used. We were particularly keen to 
distinguish untested, locally developed instruments from 
published scales which had undergone some element of 
formal testing for their predictive ability. In the current 
study, we did not aim to compare the diagnostic accur- 
acy of different instruments. 

In each hospital, data were collected on consecutive 
presentations for self-harm over a 3-month period in 
2010/2011. In total, over 6400 individuals were included. 
For the purposes of this study, self-harm was defined as 
'a deliberate non-fatal act whether physical, drug dosage 
or poisoning, carried out in the knowledge that it was 
potentially harmful and in the case of drug overdose 
that the amount taken was excessive'. 

The data collection process for self-harm episodes at 
each site was classified as a local audit and therefore 
individual patient consent was not required. 

Outcomes 

Self-harm service quality 

A service quality scale based on the Royal College of 
Psychiatrists' self-harm service standards was developed 
as part of previous work.^^ Questions on the scale per- 
tained to key aspects of service provision; for example, 
supervision arrangements for all staff involved in psycho- 
social assessments, 24 h access to specialist mental health 
services and availability of rooms for privacy and confi- 
dentiality. There were 21 items in total (see table 1 for 
all items) each scored 0-1 giving a potential scoring 
range from 0 to 21. Responses from staff at interview 
were used to complete the scales and higher scores indi- 
cated a higher overall quality of self-harm services. 
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Table 1 Service scale items with number and proportion of hospitals endorsing each item 



Item 




Yes 




number 


Service scale items 


N 


Per cent 


i 


Is there a protocol/guideline/aide memoire for staff in the A&E department for the immediate 


29 


90.6 


\ 


medical management of self-harm? 








Is there a protocol/guideline/aide memoire for staff in the A&E department for the immediate 


28 


87.5 




assessment of risk and severe mental disorder for patients who self-harm? 






3 


Is there a designated self-harm specialist clinical service? (+A&E liaison)* 


29 


90.6 


4 


Is there a local specific planning/working group (of the team who undertake the psychosocial 
assessments) which meets at least once a year to plan/oversee the service for patients who 
self-harm? 


22 


68.8 


5 


Are there psychosocial assessment training sessions for new staff who are involved in the 
psychosocial assessment of patients? 


30 


93.8 


6 


Are there supervision arrangements in place for staff members (new and existing) who 
undertake psychosocial assessments? 


23 


71.9 


7 


Are there written guidelines/a checklist, to assist psychiatric clinicians in the psychosocial 
assessment of patients who self-harm? 


27 


84.4 


8 


Does the A&E department have 24 h access to a psychiatrist, psychiatric nurse or social worker 
who is able to undertake psychosocial assessments? 


30 


93.8 


9 


If yes to 8, is immediate (within 15 min) advice available over the telephone? 


22 


68.8 


10 


If yes to 8, is emergency attendance, when requested, available within 1 h? 


7 


40.6 


11 


Do regular (at least once a year) service planning/strategy meetings take place between the 
self-harm team/psychiatric service and the general medical service involved in the care of 
patients who self-harm? 


31 


96.9 


12 


Are rooms which allow for privacy and confidentiality available for conducting interviews with 
patients who self-harm either in or close to the A&E department? 


21 


65.6 


13 


Does a formal arrangement exist with Social Services to visit and offer advice to patients who 
self-harm who have significant social difficulties? 


31 


96.9 


14 


Can those admitted as inpatients remain in hospital until they have received a psychosocial 
assessment? 


21 


65.6 


15 


Is there a policy stating that a patient's GP should be contacted within 24 h of patient discharge 
from an A&E department? 


26 


81.3 


16 


Is there a policy stating that a patient's GP should be contacted within 24 h of patient discharge 
from a medical inpatient unit? 


13 


40.6 


17 


Are patients who self-harm routinely given printed material about local services, voluntary 
groups and how to obtain access to them? 


6 


18.8 


18 


Are there any formal links with non-statutory services (eg, self-help groups, the Samaritans)? 


15 


46.9 


19 


Has a system been set up for the monitoring of hospital attendance/discharge and referral of 
patients who self-harm? 


13 


40.6 


20 


Has there been any audit of the service for patients who self-harm in the past 2 years? 


13 


40.6 



*Any liaison psychiatric service with at least one member of staff located within the ED. 
A&E, accident and emergency; ED, emergency department; GP, general practitioner. 



Repetition of self-harm 

Individuals were followed up for 6 months after their 
index episode of self-harm. Repeat self-harm episodes 
leading to hospital presentation were identified through 
hospital databases. Presentations were linked to indivi- 
duals through name, date of birth and NHS number. 

Analysis 

Our main analyses were descriptive. In order to investi- 
gate the association between use of risk scales and out- 
comes, we carried out additional analyses on aggregated 
hospital data. For this part of the analysis, we combined 
emergency department and mental health settings and 
compared hospitals using a tested tool (n=14) to hospi- 
tals not using a tested tool (n=18). This enabled us to 
capture the use of risk scales at any point along the care 



pathway. To test the robustness of the approach, we also 
carried out a sensitivity analysis based on emergency 
department assessments only. We initially used the 
Mann-Whitney U test to compare groups (services using 
and not using published risk scales) on overall service 
quality scores and repeat self-harm at 6 months. 

Using negative binomial regression models, we then 
adjusted for risk factors previously shown to influence rate 
of repetition (history of self-harm, current psychiatric treat- 
ment, cutting as method, aged under 35 and gender) .^^ 
For risk factors which were continuous variables the 
sample was split into tertiles to facilitate clinical interpret- 
ation. One hospital was missing data on the outcome of 
repeat self-harm and was excluded from the analysis. 

Statistical analysis was performed using SPSS V.20^^ 
and Stata release V.12.^^ 



Quinlivan L, Cooper J, Steeg S, etal. Open 201 4;4:e004732. doi:1 0.1 136/bmjopen-201 3-004732 



3 



Open Access 



RESULTS 

Risk and needs assessment by emergency 
department staff 

In 28 of 32 (87.5%) hospitals there was a protocol or 
guideline for the immediate assessment of suicide risk 
for patients who presented with self-harm in the emer- 
gency department. Guidelines for the assessment of 
needs were available for patients who self-harm in 7 
(21.7%) of the 32 emergency departments. There was a 
referral policy regarding who should be referred for 
assessment to the psychiatric service in 15 (46.9%) of 
the 32 emergency departments; 5 (15.6%) also had a 
written guideline or checklist to assist in deciding who 
should be referred. 

Twenty (62.5%) of the hospitals reviewed had a specific 
triage procedure for patients who presented with self- 
harm. Triage is a formal dynamic process, performed at 
initial admission to the emergency department, whereby 
patients are prioritised on the basis of clinical urgency. 
Registered nurses typically assess physical needs, patient 
history and vital signs, evaluate if emergency interven- 
tions are necessary and assign a rating based on the 
urgency of the care required.^^ In 18/32 (56.6%) hospi- 
tals there was a specific triage tool. The Manchester 
Triage tool^^ was used in 14 (43.8%) of the hospitals, 3 
(9.4%) used an in-house locally developed triage tool 
and 1 (3.1%) used the Mantis Triage tool. 



Risk tools used by emergency department staff 

Emergency department staff used a wide range of tools 
to assess risk following self-harm. Most commonly, they 
used locally developed structured proformas (n=13, 
40.6%). Staff at 12 (37.5%) emergency departments 
used published scales (table 2); 8 (66.6%) of these were 
used in conjunction with locally developed proformas or 
other generic assessments. The SAD PERSONS scale^^ 
was the most frequently used scale, being employed in 9 
(28.1%) of the emergency departments. Two (6.2%) 
hospitals used solely the SAD PERSONS scale and clin- 
ical judgment of medical staff to assess risk and staff at 1 
(3.1%) hospital reported using only clinical judgment to 
assess risk (table 2). 

Risk and needs assessment by mental health staff 

Overall, 31/32 (96.9%) mental health services reported 
that they carried out risk assessments following self- 
harm, and 29/32 (90.6%) reported evaluating service 
user's needs as part of their wider assessment process. 
Written guidelines were available to assist clinicians in 
their psychosocial assessment in 27/32 (87.5%) services. 

Risk tools used by mental health services staff 

A wide range of risk assessment tools were used by clini- 
cians in mental health services. Clinicians most fre- 
quently reported using locally developed proformas for 



Fable 2 Risk assessment tools used by emergency department and mental health staff (categories not mutually exclusive) 



Description 



Frequency of use by 
emergency departments 



Frequency of use by 
mental health services I 



Published risk scales* 



SAD PERSONS Scale^^ 


9 


2 


Suicide Intent Scale^^t 


1 


1 


Pierce Suicide Intent Scale^^ 


1 


1 


Beck Depression Inventory^^ 


1 




All other risk assessment tools 






Structured pro forma (developed locally) 


13 


22 


Galatean Risk Screening Tool (GRiST) 


1 




Emergency department mental health/suicide risk assessment form 


3 




Safe-risk pro forma 


1 




Functional analysis of care environments (FACE) 


1 




Emergency department doctors' handbook 


1 




Mental health clustering tool 


1 




Threshold assessment grid (TAG) 


2 




Policy on intranet 


1 




Risk assessment based on the care programme approach 


3 


6 


(or using CPA forms) 






CARSO: clinical assessment of risks to self and others 


1 


1 


Rapid assessment and treatment tool 


1 




St George's tool 


2 




Computerised assessment system 


1 


2 


Initial screening assessment form^ 


1 


1 


Risk assessment matrix 


2 


1 



*Published scales, which have undergone psychometric testing (eg, to evaluate reliability, predictive ability, diagnostic accuracy and construct, 
internal and external validity; Bossuyt et al, 2003).^^ 

tSection of pro forma based on Suicide Intent Scale (mental health services). 

iiTool taken from Morgan.^'' ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 
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risk assessment (n=22, 68.8%). Three (9.4%) mental 
health services used published scales in their assessment 
of self-harm. One mental health service used the Pierce 
Suicide Intent Scale^^ in conjunction with a locally 
developed proforma (3.1%). Two (6.3%) mental health 
services used the SAD PERSONS scale; one of these 
used the scale in conjunction with the Suicide Intent 
Scale^^ and a locally developed pro forma (table 2). 

Service quality score 

Fourteen of the 32 (43.8%) hospitals used published 
scales as part of risk assessment in the emergency 
department or mental health setting. Overall, service 
quality scores ranged from 10.5 to 19, with a median 
score of 14.5 (IQR 12, 16). Hospitals which used scales 
had the same median service quality score as those hos- 
pitals which did not use scales (median score (IQR): 
14.5 (12.8, 16.4) vs 14.5 (11.4, 16.0), U=121.0, p=0.90). 

Repeat self-harm at 6 months 

Hospitals which used published scales as a component of 
their risk assessments had a lower median rate of repeat 
self-harm at 6 months than hospitals which did not 
(median repeat rate (IQR): 18.5% (16.3, 20.8%) vs 22.7% 
(20.4, 25.2%), U=57.0, p=0.008). When we analysed the 
repetition data using negative binomial regression, the 
unadjusted incidence rate ratio was 1.18 (95% CI (1.00 to 
1.37), p=0.039), indicating that services which did not use 
risk scales had rates of self-harm repetition that were 
almost 20% higher than services which used such scales. 

However, when we adjusted for differences in casemix 
(by considering potential confounders such as the pro- 
portion of patients: with a history of self-harm; who used 
cutting as a method; who were in current treatment; 
who were women or aged under 35 years) the associ- 
ation between use of a risk scale and repetition was atte- 
nuated (IRR=1.13, 95% CI (0.984 to 1.29), p=0.08). 
Repeat self-harm was explained to a greater extent by 
casemix factors such as male gender (IRR=1.3, 95% CI 
(1.1 to 1.5), p=0.001 for hospitals in the highest vs lowest 
tertile for proportion of male patients) and history of 
self-harm (IRR=1.19, 95% CI (1.0 to 1.4) p=0.015 for 
hospitals in the highest vs lowest tertile for proportion of 
patients with a history of previous self-harm) . 

We conducted a sensitivity analysis to determine 
whether the effect of risk scales was specific to the emer- 
gency setting by using emergency department data only. 
The rate of repetition was lower for emergency depart- 
ments that used a published scale (19% IQR (15.1, 
21.1%) vs 22.3% (19.1, 25.1%)), although statistical evi- 
dence for a difference was limited (U=70, p=0.053) . 

DISCUSSION 
Main findings 

In this study of 32 hospitals in England we have shown 
that a wide variety of tools and scales are used for the 
assessment of risk following self-harm. There is little 



consistency in practice. Many services use locally devel- 
oped proformas or tools of questionable validity for risk 
assessments. Psychometric ally tested scales are in use in 
about one-third of services, but their use varies between 
emergency department and mental health settings. 
Overall, the SAD PERSONS is the most commonly used 
scale. Our data are descriptive and are primarily based 
on service level (rather than individual level) outcomes. 
Therefore, the results should be interpreted cautiously, 
but our findings suggest that in services where scales are 
used, the incidence of repeat self-harm may be lower. 

Strengths and limitations 

The study was observational in nature and we cannot, of 
course, infer causation. The analyses are based on aggre- 
gate data over a 3-month period in 32 hospitals; there- 
fore, generalisability may be an issue. However, we 
selected a random national sample of services and it is 
likely that the findings will be applicable to the rest of 
England. We believe our results will also be relevant to 
managed healthcare settings in other countries. Data on 
service characteristics and use of risk tools were col- 
lected through interviews, which may be subject to 
reporting bias. However, we interviewed the person most 
closely connected with services, and in all but one 
centre carried out interviews with emergency depart- 
ment and mental health staff as well as obtaining copies 
of the risk assessment tools themselves. 

We specifically restricted ourselves to scales that had 
undergone psychometric testing with some assessment 
of their predictive accuracy for suicidal behaviour or self- 
harm. Our outcome measures were self-harm repetition 
at 6 months and self-harm service provision. We did not 
record community episodes of self-harm. It is possible 
that had we done so we would not have found the appar- 
ent association between use of a scale and self-harm 
repetition. It is possible that use of the scales may have 
influenced help-seeking behaviour rather than repeti- 
tion itself. Neither did we record repeat self-harm which 
resulted in presentation to hospitals other than those in 
which the index episodes were identified. 

Interpretation and future research 

The wide range of tools in use among emergency 
department and mental health services suggests that 
there is little consensus over the best instruments for 
risk assessment. This probably reflects the poor evidence 
base and the lack of any 'gold standard' for the assess- 
ment of repeat self-harm. Staff at emergency depart- 
ments employed a particularly wide range of tools, 
which may reflect the style of service provision in that 
setting and a 'high risk' approach to management. By 
contrast, mental health staff were less likely to use pub- 
lished risk scales. This could reflect a greater reliance 
on comprehensive psychosocial assessment. 

Perhaps particularly concerning was the fact that 
locally developed tools were in widespread use in emer- 
gency department and mental health settings. This is 
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consistent with other reports/^"^^ Locally developed 
tools are generally not evidence based. They may have 
unclear scoring criteria and wide variations in structure, 
content and focus/^ They are used inconsistently 
across and within hospitals. ^^"^"^ The Royal College of 
Psychiatrists recommends that use of locally developed 
forms should be phased out altogether. 

Other services used published scales for risk assess- 
ment, but the evidence base for these is also weak. The 
psychometric properties of some scales have been investi- 
gated but their predictive value is very limited. This is 
perhaps due to the intrinsic properties of the scales them- 
selves but also because of the comparatively low base rate 
of repeat self-harm (and particularly suicide). On this 
basis, the National Institute of Health and Care 
Excellence (NICE) guideline for the longer term man- 
agement of self-harm recommended that scales should 
not be used in isolation to predict risks or determine clin- 
ical management.^ They could be used as an adjunct to 
assessment. Similarly, recent research has cast doubt on 
the use of the SAD PERSONS scale (the most commonly 
used emergency department instrument in our study) 
because of its consistently poor predictive ability. 

Interestingly, our study suggested that services which 
used risk scales may have had lower repetition rates than 
services which did not. It could be that use of the scales 
was a marker for more comprehensive services, but this 
was not borne out by our service quality scores. When 
we adjusted our models for possible confounders, the 
apparent protective effect of the use of scales was attenu- 
ated. Use of risk scales could also have reflected a more 
formal approach to assessment across the care pathway. 

Alternatively, use of the scales may have influenced 
patients' willingness to re-present as there is evidence 
from qualitative work that service users dislike 'tick box' 
approaches to assessment which may impair therapeutic 
engagement (as opposed to more holistic assessments 
which may be experienced as helpful). In other words, 
the lower rate of repetition in hospitals using tested 
scales may be because people have repeat episodes of 
self-harm in the community (which were unrecorded in 
this study), and may not return to the hospital service 
due to finding the assessment an adverse experience. A 
future study including follow-up of self-harm in the com- 
munity might help determine the influence of this on 
subsequent help-seeking behaviour. Further research on 
the positive and negative impacts of these instruments 
following self-harm would be of benefit, perhaps using 
prospective or randomised controlled designs and 
ideally with a qualitative component seeking perspectives 
of staff and patients. Until then, it is likely that the indis- 
criminant use of risk scales in clinical services will 
continue. 
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